Gender Ideologies and Language Processing: A Self-Paced Reading Time Study

1 Introduction

This document contains the results and analysis of my second experiment for my first Qualifying Paper towards the PhD in Linguistics at Stanford University¹.

2 Methods

2.1 The Experiment

The experiment analyzed herein was an online maze task study, wherein participants read a sentence such as “David is a congressperson from Virginia. He likes cycling.” For each of the 20 critical items, participants were randomly assigned to one of four conditions, the critical regions of which are enumerated and exemplified here:

1. Male-congruent: David is a congressman
2. Male-neutral: David is a congressperson
3. Female-congruent: Sally is a congresswoman
4. Female-neutral: Sally is a congressperson

Participants proceeded through the sentences by pressing a keyboard key which corresponded to the grammatical continuation of the sentence, and reading pair of grammatical and distractor items at a time. They were then asked attention check questions at the end of each sentence; attention check questions never explicitly invoked gender. Rather, they asked about the character’s home states. For a hands-on look at the experiment, you can click here to go to the same webpage particpants were directed to for the task.

##Participants We originally ran the experiment on 200 participants, recruited through the online participant recruitment platform Prolific. The mean time of the experiment was 5.39 minutes, and participants were paid $1.75 for their participation². The only restrictions placed on participants were that they:

1. Were born in the United States
2. Lived in the United States at the time of participation
3. Spoke English as a first language
4. Had not participated in the norming study for the stimuli

These requirements were implemented in order to assure that speakers came from at least somewhat similar linguistic backgrounds, as certain lexical items in the study (such as congressperson) are quite localized to the United States.

After this initial run of the experiment, we found that there was a dearth of conservative or Republican-aligned participants. As a result, we ran the experiment again, this time on 98 self-identified Republicans. This was achieved by adding a filter on Prolific so that only Republican-identified individuals could see the task. The rest of the experiment, including payment, was exactly the same, except that an additional disclaimer that participants could not use the FireFox browser experiment, after the first run revealed an incompatibility between JavaScript and FireFox. The two runs of the experiment amounted in a total of 298 participants who completed the task.

3 Prepping the Data

Before we can do much of anything with the data, we need to make sure it’s usable! This means filtering out all unimportant or extraneous trials, running exclusion criteria, and adding additional trial and item-level data that will be necessary later in the analysis.

3.1 Packages

For this analysis, we require the following packages:

library(ggplot2) 
library(tidyverse) 

Registered S3 methods overwritten by 'dbplyr':
  method         from
  print.tbl_lazy     
  print.tbl_sql      
── Attaching packages ──────────────────────────────────────────────────────────────── tidyverse 1.3.1 ──
✓ tibble  3.1.2     ✓ dplyr   1.0.6
✓ tidyr   1.1.3     ✓ stringr 1.4.0
✓ readr   1.4.0     ✓ forcats 0.5.1
✓ purrr   0.3.4     
── Conflicts ─────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()

library(lme4) 

Loading required package: Matrix

Attaching package: ‘Matrix’

The following objects are masked from ‘package:tidyr’:

    expand, pack, unpack

library(stringr)
library(languageR)
library(lmerTest)

Attaching package: ‘lmerTest’

The following object is masked from ‘package:lme4’:

    lmer

The following object is masked from ‘package:stats’:

    step

library(reshape2)

Attaching package: ‘reshape2’

The following object is masked from ‘package:tidyr’:

    smiths

source("helpers.R")

3.2 Reading in the Data

In the first instance, we need to read in the data, which has been pre-merged from both runs of the experiment. We will also in this chunk filter out all of the example trials, as well as all the data points that are from non-critical trials.

all_data <- read.csv('merged_all.csv') %>%
  filter(trial_id!= 'example') %>%
  filter(region=='critical')

3.3 Running Exclusion Criteria

Now, we want to exclude any participants who failed to answer at least 85% of the attention check questions correctly. We do this by creating a list of all participants who scored less than 85% on these checks, and then cross-referencing this list with all data points, removing any data points whose participants were in the exclusion list.

exclusion <- all_data %>% group_by(workerid) %>%
  summarise(accuracy = mean(response_correct)) %>%
  mutate(exclude = ifelse(accuracy < 0.85,'Yes','No')) %>% 
  filter(exclude == 'Yes')

all_data <- all_data[!(all_data$workerid %in% exclusion$workerid),] %>%
  filter(rt !='null')

We also want to filter out all trials in which the reading time for the critical item was more than 2.5 standard deviations from the mean reading time on that lexical item across all participants.

all_data <- all_data %>% group_by(trial_id) %>% mutate(id_mean = mean(log(rt))) %>% mutate(exclusion = (log(rt) < mean(log(rt)) - 2sd(log(rt))|(log(rt) > mean(log(rt)) + 2sd(log(rt))))) %>% ungroup() %>% filter(exclusion==FALSE)

This results in 238 trials being removed from the 5580 we got after the by-participant exclusions. We now have 5342 trials we can use for analysis.

3.4 Adding in Additional Important Trial Information

Now that we have only the rows we want, let’s add some new columns, which will contain important information for each data point. Here, we will be adding:

• Gender Ideology Subscores
• Trial Genders
• Trial Morphology Types
• Critical Item Length & Length-Controlled Residuals
• Trial Congruency
• Partipant Political Affiliation

Ideally, I would’ve added all of these but the first when I actually created the stimuli and logged responses, but I forgot to! Luckily, R allows us to do this post-hoc fairly straightforwardly… which is good, since these features will be critical in our data visualization and analysis.

3.4.1 Defining Gender Subscores

The question under investigation here is whether or not individuals’ conceptions of gender affect how they process gendered and gender-neutral forms of English personal and professional titles.

In order to examine this, we need to quanify participants’ ideological views! Here we have adopted the 13-item Social Roles Questionnaire put forth in Baber & Tucker (2006). Questions 1-5 correlate to the ‘Gender Transcendent’ subscale, and questions 6-13 correspond to the ‘Gender Linked’ subscale. Each item is scored on a scale of 0-100. So, the first thing we want to do is make two lists of columns which correspond to these two subscales, since the questions are stored individually in the data:

gender_transcendence_cols <- c('subject_information.gender_q1','subject_information.gender_q2','subject_information.gender_q3','subject_information.gender_q4','subject_information.gender_q5')

gender_linked_cols <- c('subject_information.gender_q6','subject_information.gender_q7','subject_information.gender_q8','subject_information.gender_q9','subject_information.gender_q10','subject_information.gender_q11','subject_information.gender_q12','subject_information.gender_q13')

Now we can use the mutate() method on all_data to add two new columns, one for each subscale. We tell R to take the means of the specified columns in [column_names] of all_data for each individual row: rowMeans(all_data[column_names]). We also have to subtract this mean from 100 in the case of the ‘Gender Transcendent’ subscale, since it is inversely scored. Finally, we can create an average total score regardless of subscores, simply by meaning the two subscores we already have.

all_data <- all_data %>%
  mutate(gender_trans = 100 - (rowMeans(all_data[gender_transcendence_cols]))) %>%
  mutate(gender_link = rowMeans(all_data[gender_linked_cols])) 

gender_all = c('gender_trans','gender_link')

all_data <- all_data %>%
  mutate(gender_total = rowMeans(all_data[gender_all]))

3.4.2 Adding Gender

We also want to add whether the trial included a female or male referent (but also, like, destroy the binary!). In order to do this, we’ll just add a trial_gender column that says ‘female’ if the condition was either ‘neutral_female’ or ‘congruent_female’. Otherwise, we want the trial_gender to say ‘male’.

all_data <- all_data %>%
  mutate(trial_gender = ifelse(condition=='neutral_female' | condition == 'congruent_female','female','male'))

all_data %>%
  select(workerid,rt,condition,trial_id,trial_gender)

3.4.3 Adding Morphology Type

Now we want to add whether or not the lexeme’s neutral form is developed by compounding (as in ‘congress-person’) or by the adoption of the male form (as in ‘actor’ being used more for both men and women). In this study, we only have six lexemes of the latter type, so we’ll just tell R to assign those a morph_type value of ‘adoption’ (for ‘male adoption’), and all else will be assigned a value of ‘compound’.

all_data <- all_data%>%
  mutate(morph_type = ifelse(lexeme!= 'actor' & lexeme!= 'host' & lexeme !='hunter' & lexeme!= 'villain' & lexeme!= 'heir' & lexeme!= 'hero','compound','adoption'))

all_data %>%
  select(rt,lexeme,morph_type)

3.4.4 Adding Form Length & Length-Controlled Residuals

Another important factor we want to explore is the length of the critical item! In order to add this, we simply create a new column form_length and tell R to input as that column’s value the length of the string that appears in that row’s form column, which corresponds to the orthograpic form of the critical item in that trial. Note that this will include spaces in the count!

all_data <- all_data %>%
  mutate(form_length = str_length(form))

simple_model <- lm(log(rt)~form_length, data = all_data)

all_data <- all_data %>%
  mutate(resid_rt = resid(simple_model))

summary(simple_model)

Call:
lm(formula = log(rt) ~ form_length, data = all_data)

Residuals:
    Min      1Q  Median      3Q     Max 
-0.8837 -0.3067 -0.0704  0.2480  3.1990 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 6.969210   0.023635 294.863  < 2e-16 ***
form_length 0.011369   0.002428   4.683 2.95e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4301 on 2980 degrees of freedom
Multiple R-squared:  0.007305,  Adjusted R-squared:  0.006972 
F-statistic: 21.93 on 1 and 2980 DF,  p-value: 2.954e-06

Now that we have these, we can run a simple linear regression which will show us the effect of orthographic length on reading time. Then we add a new column in the data which is the residual reading time, or the reading time in log space AFTER we control for the effects of orthographic length.

3.4.5 Adding Congruency Column

We also want to make sure we have a column which records whether or not the trial was gender-congruent (as in ‘Shelby is a congresswoman’) or gender neutral (as in ‘Shelby is a congressperson’). We add a trial_congruency column, which is valued as ‘congruent’ if that row’s condition is one of the two congruent conditions. Otherwise, it gets valued as ‘neutral’.

all_data <- all_data %>%
  mutate(trial_congruency = ifelse(condition=='congruent_male' | condition == 'congruent_female','congruent','neutral'))

3.4.6 Adding Large Political Macrocategories

Finally, we can classify participants by their particular political alignment; we can construe this broadly as “Republicans” vs. “Democrats”, with those who declined to state a preference, or placed themselves in the middle, as “Non-Partisan”.

all_data <- all_data %>%
  mutate(poli_party = ifelse(subject_information.party_alignment == 1 | subject_information.party_alignment == 2,'Republican',ifelse(subject_information.party_alignment == 4 | subject_information.party_alignment == 5,'Democrat','Non-Partisan')))

4 Data Visualization

Now we can start analysing the data by means of data visualization.

inauguration_2021 = c("#5445b1", "#749dae", "#f3c483", "#5c1a33", "#cd3341","#f7dc6a")

4.1 Ideology by Age

ggplot(all_data, aes(x=subject_information.age, y=gender_total)) + 
  geom_point(alpha=.5) + 
  geom_smooth(method = 'lm', size=1.2)

4.2 Reading Time by Gender Ideology

ggplot(all_data, aes(x=gender_total, y=resid_rt, color=trial_congruency)) + 
  geom_point(alpha=.5) + 
  geom_smooth(method = 'lm', size=1.2) + 
  theme_minimal()

4.3 Reading Time by Age

4.4 Reading Time by Congruency and Gender

agg_speaker_mean_con <- all_data %>%
  group_by(condition,workerid) %>%
  summarize(MeanRT=mean(resid_rt))

`summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.

all_data %>%
  group_by(condition,trial_gender) %>%
  summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
  mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
  ggplot(aes(x=condition,y=MeanRT,color=trial_gender)) + 
  geom_point(size=3) + 
  geom_jitter(data = agg_speaker_mean_con, aes(y=MeanRT),alpha=.1,color='darkred') + 
  geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) + 
  scale_color_manual(values = inauguration_2021) + 
  theme_minimal()

`summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.

4.5 Reading Time by Congruency, Gender, and Lexical Item

all_data %>%
  group_by(condition,trial_gender,trial_congruency,lexeme) %>%
  summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
  mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
  ggplot(aes(x=condition,y=MeanRT,color=trial_gender,shape=trial_congruency)) + 
  geom_point(size=3) +
  geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) + 
  facet_wrap(~ lexeme) +
  theme(axis.text.x = element_text(angle = 45, vjust = .7, hjust=.7)) + 
  scale_color_manual(values = inauguration_2021) + 
  facet_wrap(~lexeme)

`summarise()` has grouped output by 'condition', 'trial_gender', 'trial_congruency'. You can override using the `.groups` argument.

4.5.1 Male Adoptions Only

all_data %>%
  filter(morph_type == "adoption") %>%
  group_by(condition,trial_gender,trial_congruency,lexeme) %>%
  summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
  mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
  ggplot(aes(x=condition,y=MeanRT,color=trial_gender,shape=trial_congruency)) + 
  geom_point(size=3) +
  geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) + 
  facet_wrap(~ lexeme) +
  theme(axis.text.x = element_text(angle = 45, vjust = .7, hjust=.7)) + 
  scale_color_manual(values = inauguration_2021) + 
  facet_wrap(~lexeme)

`summarise()` has grouped output by 'condition', 'trial_gender', 'trial_congruency'. You can override using the `.groups` argument.

4.5.2 Compounds only

all_data %>%
  filter(morph_type == "compound") %>%
  group_by(condition,trial_gender,trial_congruency,lexeme) %>%
  summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
  mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
  ggplot(aes(x=condition,y=MeanRT,color=trial_gender,shape=trial_congruency)) + 
  geom_point(size=3) +
  geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) + 
  facet_wrap(~ lexeme) +
  theme(axis.text.x = element_text(angle = 45, vjust = .7, hjust=.7)) + 
  scale_color_manual(values = inauguration_2021) + 
  facet_wrap(~lexeme)

`summarise()` has grouped output by 'condition', 'trial_gender', 'trial_congruency'. You can override using the `.groups` argument.

4.6 Raw Reading Time by Trial Gender

temp <- all_data %>%
  group_by(trial_gender) %>%
  summarize(MeanRT = mean(rt), CI.Low = ci.low(rt), CI.High = ci.high(rt)) %>%
  mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High)

dodge = position_dodge(.9)
ggplot(data=temp, aes(x=trial_gender,y=MeanRT,fill=trial_gender)) + 
  geom_bar(stat='identity',position=dodge) + 
  geom_errorbar(aes(ymin=YMin,ymax=YMax),width=.25,position=dodge) +
  theme(legend.position = 'none')

dodge = position_dodge(.9)

all_data %>% 
  group_by(trial_gender) %>%
  summarize(MeanRT = mean(resid_rt)) %>%
  ggplot(aes(x=trial_gender,y=MeanRT,fill=trial_gender)) + 
  geom_bar(stat='identity',position=dodge) + 
  theme(legend.position = 'none')

4.7 Reading Time Residuals by Congruency

dodge = position_dodge(.9)

all_data %>% 
  group_by(trial_congruency) %>%
  summarize(MeanRT = mean(resid_rt)) %>%
  ggplot(aes(x=trial_congruency,y=MeanRT,fill=trial_congruency)) + 
  geom_bar(stat='identity',position=dodge) + 
  theme(legend.position = 'none')

4.8 My Morphological Type and Gender

all_data %>%
  group_by(condition,trial_gender,morph_type) %>%
  summarize(MeanRT = mean(resid_rt), CI.Low = ci.low(resid_rt), CI.High = ci.high(resid_rt)) %>%
  mutate(YMin = MeanRT - CI.Low, YMax = MeanRT + CI.High) %>%
  ggplot(aes(x=condition,y=MeanRT,color=trial_gender)) + 
  geom_point(size=3) + 
  geom_errorbar(aes(ymin=YMin,ymax=YMax), width=.25) + 
  scale_color_manual(values = inauguration_2021) + 
  facet_wrap(~morph_type) + 
  theme(axis.text.x = element_text(angle=45, vjust = 0.5))

`summarise()` has grouped output by 'condition', 'trial_gender'. You can override using the `.groups` argument.

4.9 Task Familiarity

agg_speaker_trial <- all_data %>%
  group_by(condition,workerid) %>%
  summarize(MeanRT=mean(resid_rt))

`summarise()` has grouped output by 'condition'. You can override using the `.groups` argument.

5 Model Analyses

6 Testing Models

test_model <- lm(resid_rt~trial_congruency*morph_type, data=all_data)

summary(test_model)

Call:
lm(formula = resid_rt ~ trial_congruency * morph_type, data = all_data)

Residuals:
    Min      1Q  Median      3Q     Max 
-0.9005 -0.3021 -0.0725  0.2436  3.1904 

Coefficients:
                                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)                                 0.04198    0.02044   2.054 0.040111 *  
trial_congruencyneutral                    -0.03333    0.02864  -1.164 0.244666    
morph_typecompound                         -0.10134    0.02434  -4.164 3.22e-05 ***
trial_congruencyneutral:morph_typecompound  0.13118    0.03422   3.834 0.000129 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4279 on 2978 degrees of freedom
Multiple R-squared:  0.01097,   Adjusted R-squared:  0.009969 
F-statistic: 11.01 on 3 and 2978 DF,  p-value: 3.478e-07

test_model2 <- lm(resid_rt~trial_gender, data=all_data)

summary(test_model2)

Call:
lm(formula = resid_rt ~ trial_gender, data = all_data)

Residuals:
    Min      1Q  Median      3Q     Max 
-0.8799 -0.3054 -0.0716  0.2498  3.1953 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)
(Intercept)      -0.003744   0.011138  -0.336    0.737
trial_gendermale  0.007488   0.015751   0.475    0.635

Residual standard error: 0.4301 on 2980 degrees of freedom
Multiple R-squared:  7.583e-05, Adjusted R-squared:  -0.0002597 
F-statistic: 0.226 on 1 and 2980 DF,  p-value: 0.6345

compounds_only <- all_data %>%
  filter(morph_type == 'compound')

compounds_only <- compounds_only %>%
  mutate(ctrial_congruency = as.numeric(as.factor(trial_congruency))-mean(as.numeric(as.factor(trial_congruency)))) %>%
  mutate(ctrial_gender = as.numeric(as.factor(trial_gender))-mean(as.numeric(as.factor(trial_gender)))) %>%
  mutate(cgender_link = scale(gender_link)) %>%
  mutate(cgender_total = scale(gender_total))

compound_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_total + (1|workerid) + (1|lexeme) + (1|name),data = compounds_only)

boundary (singular) fit: see ?isSingular

summary(compound_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_total +  
    (1 | workerid) + (1 | lexeme) + (1 | name)
   Data: compounds_only

REML criterion at convergence: 1516.9

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.9463 -0.6196 -0.1141  0.5120  5.1743 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.072802 0.26982 
 name     (Intercept) 0.000000 0.00000 
 lexeme   (Intercept) 0.009709 0.09853 
 Residual             0.097840 0.31279 
Number of obs: 2089, groups:  workerid, 152; name, 24; lexeme, 14

Fixed effects:
                                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                     -0.01169    0.03492   33.77863  -0.335   0.7398    
ctrial_congruency                                0.10120    0.01386 1927.09399   7.301 4.16e-13 ***
ctrial_gender                                    0.02769    0.01387 1927.54232   1.996   0.0461 *  
cgender_total                                    0.05201    0.02294  150.06294   2.267   0.0248 *  
ctrial_congruency:ctrial_gender                 -0.01774    0.02780 1929.12259  -0.638   0.5234    
ctrial_congruency:cgender_total                  0.01692    0.01386 1926.12747   1.221   0.2224    
ctrial_gender:cgender_total                     -0.02024    0.01389 1927.94098  -1.457   0.1452    
ctrial_congruency:ctrial_gender:cgender_total   -0.04153    0.02776 1927.38754  -1.496   0.1349    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
                 (Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc       0.000                                                                
ctrial_gndr       0.000 -0.008                                                         
cgender_ttl       0.000  0.003 -0.001                                                  
ctrl_cngrncy:ct_ -0.002  0.001  0.003  0.001                                           
ctrl_cngrncy:cg_  0.002 -0.006  0.003  0.009 -0.002                                    
ctrl_gndr:_      -0.001  0.005  0.004  0.000  0.016           -0.003                   
ctrl_cn:_:_       0.001  0.001  0.016 -0.001  0.000            0.000            0.034  
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular

adoptions_only <- all_data %>%
  filter(morph_type == 'adoption')

adoptions_only <- adoptions_only %>%
  mutate(ctrial_congruency = as.numeric(as.factor(trial_congruency))-mean(as.numeric(as.factor(trial_congruency)))) %>%
  mutate(ctrial_gender = as.numeric(as.factor(trial_gender))-mean(as.numeric(as.factor(trial_gender)))) %>%
  mutate(cgender_link = scale(gender_link)) %>%
  mutate(cgender_total = scale(gender_total))

adoptions_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_total + (1|workerid) + (1|lexeme) + (1|name),data = adoptions_only)

boundary (singular) fit: see ?isSingular

summary(adoptions_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_total +  
    (1 | workerid) + (1 | lexeme) + (1 | name)
   Data: adoptions_only

REML criterion at convergence: 780.7

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.2764 -0.6128 -0.1019  0.4977  7.0825 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.05999  0.2449  
 name     (Intercept) 0.00000  0.0000  
 lexeme   (Intercept) 0.02594  0.1611  
 Residual             0.10261  0.3203  
Number of obs: 893, groups:  workerid, 152; name, 24; lexeme, 6

Fixed effects:
                                                Estimate Std. Error         df t value Pr(>|t|)   
(Intercept)                                     0.026404   0.069521   5.924863   0.380  0.71734   
ctrial_congruency                              -0.026661   0.022600 773.237762  -1.180  0.23848   
ctrial_gender                                  -0.024247   0.022594 773.262319  -1.073  0.28353   
cgender_total                                   0.061983   0.022598 148.801276   2.743  0.00684 **
ctrial_congruency:ctrial_gender                -0.033364   0.045588 779.086714  -0.732  0.46447   
ctrial_congruency:cgender_total                -0.009892   0.022408 764.311685  -0.441  0.65901   
ctrial_gender:cgender_total                    -0.011764   0.022586 769.611878  -0.521  0.60261   
ctrial_congruency:ctrial_gender:cgender_total   0.039454   0.045136 767.084621   0.874  0.38232   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
                 (Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc       0.000                                                                
ctrial_gndr       0.000  0.029                                                         
cgender_ttl      -0.001 -0.008  0.002                                                  
ctrl_cngrncy:ct_  0.003  0.001  0.005 -0.008                                           
ctrl_cngrncy:cg_ -0.003 -0.021 -0.013 -0.031  0.020                                    
ctrl_gndr:_       0.001 -0.013  0.002  0.003 -0.005            0.007                   
ctrl_cn:_:_      -0.003  0.020 -0.003  0.001 -0.007            0.011           -0.057  
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular

6.1 Full Model

all_data <- all_data %>%
  mutate(ctrial_congruency = as.numeric(as.factor(trial_congruency))-mean(as.numeric(as.factor(trial_congruency)))) %>%
  mutate(ctrial_gender = as.numeric(as.factor(trial_gender))-mean(as.numeric(as.factor(trial_gender)))) %>%
  mutate(cgender_link = scale(gender_link)) %>%
  mutate(cgender_total = scale(gender_total)) %>%
  mutate(cmorph_type = as.numeric(as.factor(morph_type))-mean(as.numeric(as.factor(morph_type))))

complex_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_total + (1|workerid) + (1|lexeme) + (1|name),data = all_data)

boundary (singular) fit: see ?isSingular

summary(complex_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_total +  
    (1 | workerid) + (1 | lexeme)
   Data: all_data

REML criterion at convergence: 2129

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.5602 -0.6318 -0.1277  0.5143  7.7057 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.06833  0.2614  
 lexeme   (Intercept) 0.01362  0.1167  
 Residual             0.10093  0.3177  
Number of obs: 2982, groups:  workerid, 152; lexeme, 20

Fixed effects:
                                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                   -1.370e-04  3.413e-02  4.764e+01  -0.004   0.9968    
ctrial_congruency                              5.971e-02  1.166e-02  2.806e+03   5.120 3.27e-07 ***
ctrial_gender                                  9.664e-03  1.166e-02  2.806e+03   0.829   0.4075    
cgender_total                                  5.441e-02  2.199e-02  1.500e+02   2.474   0.0145 *  
ctrial_congruency:ctrial_gender               -1.767e-02  2.334e-02  2.807e+03  -0.757   0.4490    
ctrial_congruency:cgender_total                1.010e-02  1.167e-02  2.807e+03   0.865   0.3870    
ctrial_gender:cgender_total                   -1.820e-02  1.168e-02  2.807e+03  -1.559   0.1191    
ctrial_congruency:ctrial_gender:cgender_total -1.689e-02  2.335e-02  2.807e+03  -0.723   0.4695    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
                 (Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc       0.000                                                                
ctrial_gndr       0.000  0.000                                                         
cgender_ttl      -0.001  0.001 -0.001                                                  
ctrl_cngrncy:ct_  0.000  0.000  0.001  0.000                                           
ctrl_cngrncy:cg_  0.001 -0.001 -0.003  0.001 -0.003                                    
ctrl_gndr:_       0.000 -0.001  0.001  0.000  0.002           -0.001                   
ctrl_cn:_:_       0.000 -0.001  0.003  0.000  0.001            0.000            0.004  

6.2 Gender Total

ideology_model <- lmer(resid_rt~ctrial_congruency*ctrial_gender*cgender_link + (1|workerid) + (1|lexeme) + (1|name), data=all_data)

boundary (singular) fit: see ?isSingular

summary(ideology_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: resid_rt ~ ctrial_congruency * ctrial_gender * cgender_link +  
    (1 | workerid) + (1 | lexeme) + (1 | name)
   Data: all_data

REML criterion at convergence: 2129.1

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.5813 -0.6321 -0.1304  0.5107  7.6659 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.06754  0.2599  
 name     (Intercept) 0.00000  0.0000  
 lexeme   (Intercept) 0.01365  0.1168  
 Residual             0.10099  0.3178  
Number of obs: 2982, groups:  workerid, 152; name, 24; lexeme, 20

Fixed effects:
                                               Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                  -1.209e-04  3.407e-02  4.720e+01  -0.004  0.99718    
ctrial_congruency                             5.968e-02  1.167e-02  2.806e+03   5.116 3.33e-07 ***
ctrial_gender                                 9.695e-03  1.167e-02  2.806e+03   0.831  0.40607    
cgender_link                                  6.112e-02  2.184e-02  1.500e+02   2.798  0.00581 ** 
ctrial_congruency:ctrial_gender              -1.761e-02  2.334e-02  2.807e+03  -0.754  0.45070    
ctrial_congruency:cgender_link                1.229e-02  1.167e-02  2.807e+03   1.053  0.29254    
ctrial_gender:cgender_link                   -1.021e-02  1.168e-02  2.807e+03  -0.874  0.38218    
ctrial_congruency:ctrial_gender:cgender_link -6.519e-03  2.336e-02  2.807e+03  -0.279  0.78017    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
                 (Intr) ctrl_c ctrl_g cgndr_ ctrl_cngrncy:ct_ ctrl_cngrncy:cg_ ctrl_g:_
ctrl_cngrnc       0.000                                                                
ctrial_gndr       0.000  0.000                                                         
cgender_lnk       0.000  0.001 -0.001                                                  
ctrl_cngrncy:ct_  0.000  0.000  0.001  0.000                                           
ctrl_cngrncy:cg_  0.001 -0.001 -0.002  0.001 -0.002                                    
ctrl_gndr:_       0.000 -0.001  0.000  0.000  0.002           -0.001                   
ctrl_cn:_:_       0.000  0.000  0.003  0.000  0.000            0.000            0.004  
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular

7 Comments

all_data %>%
  group_by(workerid,poli_party) %>%
  summarise(comments = paste(unique(subject_information.comments))) %>%
  select(poli_party,comments)

`summarise()` has grouped output by 'workerid'. You can override using the `.groups` argument.
Adding missing grouping variables: `workerid`

8 Testing Grounds

8.1 Checking the times

new_toy <- read.csv('maze_task_1-merged.csv') %>%
  filter(trial_id!= 'example') %>%
  filter(response_correct != 0) %>%
  mutate(trial_gender = ifelse(condition=='neutral_female' | condition == 'congruent_female','female','male')) %>%
  mutate(trial_congruency = ifelse(condition=='congruent_male' | condition == 'congruent_female','congruent','neutral'))

new_toy %>%
  group_by(word_idx,trial_gender) %>%
  summarize(MeanRT = mean(rt)) %>%
  ggplot(aes(x=word_idx, y=log(MeanRT), color=trial_gender)) + 
  geom_line() + 
  geom_point()

`summarise()` has grouped output by 'word_idx'. You can override using the `.groups` argument.

new_toy %>%
  group_by(word_idx,trial_gender) %>%
  summarize(MeanRT = mean(rt)) %>%
  ggplot(aes(x=word_idx, y=MeanRT, color=trial_gender)) + 
  geom_line() + 
  geom_point()

`summarise()` has grouped output by 'word_idx'. You can override using the `.groups` argument.

Rscript merge_results.R maze_task_1-merged.csv dems maze_task_2-merged.csv reps

---

1. Part of this experiment and analysis was also carried out as part of my class project in Stanford’s LINGUIST 245B ‘Methods in Psycholinguistics’ class, taught by Judith Degen.↩︎

2. This amounts to an hourly rate of $20.73. We originally anticipated that participants would take an average of 7 minutes to complete the experiment, and set the base pay at $15 an hour.↩︎